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EXECUTIVE  SUMMARY 


The  Area  Air  Defense  Commander  (AADC)  plans,  coordinates,  and  executes  air  defense  opera¬ 
tions  in  a  theater  of  operations.  This  responsibility  requires  a  clear  tactical  picture  of  friendly  and 
hostile  forces  and  their  capabilities.  A  new  display  has  been  prototyped  for  this  purpose  (figure  1). 
This  prototype  display  has  a  three-dimensional  (3-D)  perspective  format  and  uses  novel  volumetric 
(realistic)  symbols.  It  has  been  touted  as  enabling  users  to  become  more  rapidly  aware  of  the  tactical 
picture  than  a  comparable  two-dimensional  (2-D)  display.  This  study  addresses  two  questions; 

1.  Does  the  3-D  display  format  promote  more  rapid  Situation  Awareness  (SA)  than  a  conventional 
2-D  display  format? 

2.  Do  the  new  detailed  3-D  volumetric  realistic  symbols  enhance  performance  over  their  more 
conventional  non-reahstic  counterparts? 

Three  participant  groups  observed  a  9-minute  scenario  in  three  different  ways.  The  first  group  saw 
the  scenario  in  a  3-D  perspective  view  with  realistic  s5unbols.  The  second  group  saw  a  2-D  top-down 
view  with  reaUstic  symbols.  The  third  group  saw  a  2-D  top-down  view  with  conventional  (non- 
realistic)  symbols.  The  scenario  was  stopped  every  30  seconds  in  each  condition  and  an  online  ques¬ 
tionnaire  assessed  participant  SA  attributes  of  depicted  tracks. 

FINDINGS 

•  Participants  became  increasingly  aware  of  the  track  identities  over  time,  but  they  had  difficulty 
remembering  the  spatial  attributes  of  those  tracks. 

•  2-D  SA  was  initially  2.5  times  higher  than  3-D  SA  and  remained  significantly  higher  for  the 
first  4  minutes. 

•  3-D  SA  increased  six  times  faster  than  2-D  SA  for  the  first  5  minutes. 

•  Overall  SA  was  equivalent  for  2-D  and  3-D  after  9  minutes. 

•  3-D  was  superior  to  2-D  for  the  attributes  of  altitude  (4  times)  and  attitude  (20  times). 

•  Realistic  symbols  were  superior  (two  times)  to  non-realistic  symbols  for  the  heading  attribute. 

•  Participants  found  it  harder  to  learn  realistic  symbology. 

CONCLUSIONS 

•  2-D  top-down  display  with  conventional  (non-realistic)  symbols  provided  superior  SA  during 
the  first  4  minutes. 

•  3-D  displays  with  reahstic  symbols  provided  superior  SA  for  altitude,  attitude,  and  heading 
attributes. 
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RECOMMENDATIONS 

•  Consider  using  2-p  top-down  displays  with  conventional  (non-realistic)  symbols  when  very 
rapid  SA  is  necessary. 

•  Consider  using  2-D  or  3-D  realistic  symbology  when  rapid  heading  appreciation  is  necessary. 

•  Consider  using  3-D  displays  when  rapid  altitude  and  attitude  appreciation  outweighs  other 
factors. 
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INTRODUCTION 


The  Area  Air  Defense  Commander  (AADC)  plans,  coordinates,  and  executes  air  defense  opera¬ 
tions  in  a  theater  of  operations.  This  responsibility  requires  a  clear  tactical  picture  of  friendly  and 
hostile  forces  and  their  capabilities.  A  new  display  has  been  prototyped  for  this  purpose  (figure  1). 
This  prototype  display  has  a  three-dimensional  (3-D)  perspective  format  and  uses  novel  volumetric 
(realistic)  symbols.  It  has  been  touted  as  enabling  users  to  become  more  rapidly  aware  of  the  tactical 
picture  than  a  comparable  two-dimensional  (2-D)  display.  This  study  addresses  two  questions: 

1.  Does  the  3-D  display  format  promote  more  rapid  Situation  Awareness  (SA)  than  a  conventional 
2-D  display  format? 

2.  Do  the  new  detailed  3-D  volumetric  realistic  symbols  enhance  performance  over  their  more 
conventional  non-realistic  counterparts? 


Figure  1.  Screenshot  from  early  version  of  AADC  display  prototype  showing  its  realistic 
symbols  in  3-D. 

2-D  VERSUS  3-D  DISPLAY  FORMAT 

Conventional  2-D  “bird’s  eye”  displays  represent  two  dimensions  of  the  earth’s  surface  and  leave 
the  third  dimension  (altitude)  implicit.  3-D  perspective  displays,  on  the  other  hand,  try  to  show  the 
third  dimension  as  well  by  depicting  the  perspective  projection  of  an  air  space  from  a  certain  point  of 
view  above  the  ground  plane  onto  the  display  screen  (e.g.,  Ellis,  McGreevy,  and  Hitchcock,  1987). 

Why  consider  changing  from  a  2-D  to  a  3-D  perspective  display  format?  There  are  four  main  rea¬ 
sons.  The  first  reason  is  to  present  users  with  a  visual  representation  of  depth  on  their  inherently  flat 
2-D  displays.  Second,  3-D  displays  may  be  more  natural  for  users.  As  we  view  objects,  our  retinal 
images  are  perspective  projections  of  the  world,  so  perspective  displays  may  be  inherently  more 
ecologically  plausible  (Warren  and  Wertheim,  1990).  Others  have  argued  that  because  we  plan. 
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perceive,  and  act  in  a  3-D  environment,  the  integrated  format  of  3-D  displays  should  make  them 
easier  to  operate  (Wickens,  1992).  Third,  users  want  3-D  displays.  Users  preferred  the  AADC  proto¬ 
type  display  during  operational  trials  (Kramer,  Hontz,  and  Broyles,  1995*,  cited  in  Schiller,  Mitchell, 
and  Bemis,  1998  ).  Fourth,  and  most  importantly,  some  argue  that  3-D  displays  give  users  a  clearer 
picture  of  battlefield  tactical  information  than  conventional  2-D  displays  (e.g.,  Dennehy  et  al.,  1994). 

By  conveying  all  dimensions  of  space  directly,  3-D  displays  may  provide  a  clearer  picture  because 
users  are  saved  from  having  to  slowly  search  through  text  boxes  of  airtrack  attributes  found  on 
conventional  2-D  displays  (Dennehy  et  al.,  1994).  Supporting  this  notion,  Baumann,  Blanksteen,  and 
Dennehy  (1997)  showed  that  users  found  descending  planes  more  rapidly  in  a  3-D  perspective  view 
display  than  in  a  2-D  top-down  display.  This  finding  was  no  surprise  to  expert  users.  Eddy,  Kribs, 
and  Cowen  (1999)  asked  surface  warfare  operators  what  they  thought  the  limitations  of  the  current 
displays  were  and  what  they  want  included  in  the  next-generation  tactical  displays.  They  reported 
that  the  experts  rated  the  everyday  task  of  recognizing  and  classifying  elements  of  air,  surface,  and 
subsurface  threats  as  very  difficult  with  existing  display  technologies.  One  respondent  expressed  the 
following:  “We  hate  digital  readouts  because  it  takes  longer  to  read  and  it  doesn’t  tell  a  story.”  Most 
respondents  indicated  a  need  for  a  console  “representing  these  sort  of  things  in  some  other  visual  way 
that  more  intuitively  paints  a  picture.” 

Despite  these  cogent  arguments  for  3-D,  we  should  also  look  at  the  disadvantages.  It  may  not  be 
wise  to  base  decisions  on  the  fact  that  people  like  the  look  of  3-D  displays.  We  know  that  user  pref¬ 
erence  and  performance  do  not  always  correlate  well  (Bailey,  1993).  In  fact,  sometimes  users  want 
what  is  not  best  for  them  (Andre  and  Wickens,  1995).  For  example,  Carswell  (1991)  showed  that 
people  rate  3-D  graphs  and  bar-charts  as  more  impressive  than  2-D  when  at  the  same  time  they  were 
less  accurate  at  telling  what  the  data  depicted  in  3-D  compared  to  2-D.  There  is  a  big  price  to  pay 
when  one  moves  from  2-D  to  3-D.  When  a  third  dimension  is  represented  in  a  2-D  space,  attributes 
along  all  three  dimensions  become  ambiguous  in  the  following  ways: 

1.  They  are  prone  to  distortion.  To  perceive  the  3-D  arrangement  of  any  scene,  the  visual  system 
must  perform  “reverse-optics”  to  figure  out  what  objects  plausibly  gave  rise  to  the  retinal  images  of 
that  scene.  This  is  difficult  because  many  different  possible  3-D  worlds  can  produce  the  same  retinal 
images.  The  visual  system  must  make  its  best  guess  as  to  what  goes  where.  In  2-D  displays, 
everything  is  put  in  the  background  plane.  In  3-D  displays,  distortions  readily  occur  because  it  is 
difficult  to  correctly  place  objects  along  the  line  of  sight.  There  is  usually  only  a  restricted  set  of 
depth  cues  available  to  facilitate  correct  assignment.  Additional  depth  cue  features  that  disambiguate 
depth,  such  as  drop-lines  or  drop-shadows  from  an  aircraft  to  the  ground  plane,  uniquely  sp^ify  2-D 
location  (Wickens,  1992).  The  AADC  prototype  display  uses  drop-shadows. 

2.  They  are  prone  to  clutter.  Use  of  drop-shadows  or  other  augmentations  to  clarify  depth 
increases  the  number  of  symbols  in  the  display.  These  augmentations  only  work  if  users  can  maintain 


Kramer,  T.  R.,  E.  B.  Hontz,  and  J.  W.  Broyles.  1995.  "Force  Threat  Evaluation  and  Weapons  Assignment 
(FTEWA)  System  Operational  Suitability  and  Human  Factors  Evaluation  for  USS  Normandy  (CG60)  Adriatic 
Seas  Red  Crown  Operations.  Internal  documentation.  Human  Systems  Interface  Technology  Research  Code 
D44215,  SSC  San  Diego,  CA. 

^  Schiller,  E.,  C.  Mitchell,  and  S.  Bemis.  1998.  "AADC  3-D  Perspective  Display  Evaluation:  Study  on  Resolution 
and  Monitor  Type."  Internal  documentation.  Human  Systems  Interface  Technology  Research,  Code  D44215 
SSC  San  Diego,  CA. 
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“perceptual  links”  between  them  and  the  symbols  for  the  aircraft.  In  dense  displays,  users  may  have 
too  many  links  to  remember  them  all.  Baumann  et  al.  (1997)  found  it  surprising  that  users  failed  to 
find  the  last  couple  of  descending  planes  in  a  high-density  AADC  prototype-like  simulation 
when  compared  with  2-D  control  ^splays.  Burnett  and  Barfield  (1991)  found  that  a  3-D  advantage 
over  2-D  for  low-density  displays  disappeared  at  higher  densities. 

3.  They  are  poor  for  precision  tasks.  St.  John  and  Cowen  (1999)  recently  showed  that  although 
3-D  displays  conveyed  shape  well,  they  were  poor  for  tasks  that  required  them  to  determine  distances 
and  directions  between  objects  in  3-D.  An  AADC  may  not  need  precise  spatial  information,  but  must 
have  a  good  understanding  of  the  tactical  picture. 

4.  Realistic  icons  do  not  scale  well.  Emmert’s  law  of  size  constancy  states  that  perceived  distance 
is  directly  related  to  image  size  (see  McKee  and  Smallman,  1998).  In  a  3-D  display,  distant  icons 
should  be  smaller,  but  they  will  quickly  get  too  small  to  be  recognized.  Following  Emmert’s  law, 
falsely  scaled  icons  may  be  perceived  at  the  wrong  depth  in  this  display. 

5.  They  are  not  isotropic.  2-D  displays  give  an  equal  display  area  to  an  equal  physical  area  on  the 
ground  plane.  In  this  sense,  they  are  said  to  be  isotropic.  However,  because  of  imaging  geometry, 

3-D  perspective  displays  must  give  less  display  area  to  regions  near  the  horizon  and  vanishing  point. 
In  addition  to  a  loss  of  fidelity,  this  makes  clutter  even  more  problematic  because  far  objects  and 
their  drop-shadows  will  be  packed  into  an  even  smaller  display  area. 

Consequently,  current  research  has  been  equivocal  at  documenting  actual  performance  improve¬ 
ments  with  3-D  displays.  Across  an  array  of  tasks,  some  studies  have  found  3-D  benefits  over  2-D 
(Andre  et  al.,  1991;  Bemis,  Leeds,  and  Winer,  1988;  Burnett  and  Barfield,  1991;  Ellis  et  al.,  1987; 
Haskell  and  Wickens,  1993;  VanBreda  and  Veltman,  1998;  Wickens  and  Prevett,  1995;  Ware  and 
Franck,  1994;  Baumann  et  al.,  1997),  some  have  found  rough  parity  (Wickens,  Liang,  Prevett,  and 
Olmos,  1996;  Wickens  and  May,  1994),  and  some  have  found  2-D  superior  to  3-D  (Boyer  and  Wick¬ 
ens,  1994;  Boyer  et  al.,  1995;  Endsley,  1995a;  O'Brien  and  Wickens,  1997;  Wickens  et  al.  1995). 
Several  factors  contribute  to  this  confusion:  (1)  different  tasks  and  displays  used  in  different  studies, 
(2)  3-D  displays  typically  different  in  the  specification  of  imaging  geometry  and  depth  cues,  and  (3) 
comparisons  between  3-D  and  2-D  displays  not  matched  as  attribute  information. 

TRACK  SYMBOLS 

In  addition  to  the  issue  of  2-D  versus  3-D,  we  must  know  how  to  best  represent  the  military 
symbology  in  the  display.  As  noted  above,  the  AADC  prototype  display  uses  realistic  3-D  volu¬ 
metric  icons  to  represent  each  t5q)e  of  aircraft  and  ship.  These  icons  seem  intuitive,  but  do  they 
lead  to  better  SA?  Consider  the  various  depictions  of  a  bomber  in  figure  2.  The  left  of  figure  2 
shows  a  3-D  realistic  rendition  of  a  bomber.  The  center  shows  a  realistic  bomber  in  2-D,  and  the 
right  shows  a  conventional  (non-realistic)  symbol  for  a  bomber.  Which  is  better?  Florence  and 
Geiselman  (1986)  examined  visual  search  for  symbols  depicted  realistically  (iconicaUy)  versus 
non-realistically  (s5mibolically)  on  a  2-D  display.  They  found  a  slight  advantage  for  iconic  pres¬ 
entation. 
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Figure  2.  Three  different  ways  of  depicting  bomber  headed  east  on  display.  Bomber  depicted  realistically 
in  3-D  as  used  in  AADC  prototype  display  (left),  realistically  in  2-D  (center),  and  symbolically  in  2-D  (right) 
(e.g.,  Mil-Std-2525B). 

OBJECTIVE 

This  objective  this  study  was  to  test  whether  a  3-D  display  format  promotes  more-rapid  SA 
than  a  conventional  2-D  display  format  and  if  realistic  symbols  are  associated  with  better 
performance  than  conventional  2-D  symbols. 

APPROACH 

We  employed  three  different  display  conditions:  (1)  a  3-D  realistic  condition,  similar  to  the  AADC 
prototype  display,  with  realistic  icons,  (2)  a  2-D  realistic  condition  that  uses  realistic  icons  on  a  2-D 
display,  and  (3)  a  2-D  non-realistic  condition  that  uses  conventional  symbols  on  a  2-D  display.  We 
are  concerned  with  how  the  conditions  affect  element  perception  in  a  depicted  situation  (i.e.,  how  it 
affects  their  SA).  SA  has  been  defined  as  “the  perception  of  the  elements  in  the  environment  within  a 
volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the  projection  of  their  status  on 
the  near  future”  (Endsley,  1988,  p.  97).  It  is  considered  vital  for  good  performance  in  Air  Traffic 
Control  (ATC)  tasks  and  air  combat  simulations  (Endsley,  1995a).  SA  is  closely  related  to  the  mental 
model  that  a  user  develops  of  the  display  information  (Mogford,  1997;  Sarter  and  Woods,  1991). 
Endsley  (1995b)  has  proposed  the  following  three-step  model  of  SA:  (1)  successful  SA  involves  per¬ 
ceiving  the  current  situation  elements  (level  1  SA),  (2)  comprehending  how  those  elements  figure 
into  the  current  situation  (level  2  SA),  and  (3)  figuring  how  the  future  situation  may  develop  from  the 
present  one  (level  3  SA).  Level  1  SA  is  the  most-critical  SA  level.  Without  correct  perception  of 
what  is  depicted  in  a  display,  decision-making  will  be  based  on  faulty  or  sketchy  information.  We 
focused  on  Level  1  SA. 

Various  ways  of  measuring  SA  have  been  proposed.  Among  the  better  techniques  are  online  ques¬ 
tionnaires.  Questionnaires  are  good  in  that  they  are  objective  and  are  touted  as  minimally  corrupting 
the  awareness  that  they  are  intended  to  measure  (Endsley,  1995a).  They  have  been  used  successfully 
in  several  studies.  Marshak  et  al.  (1987)  first  used  questionnaires  when  they  queried  participants 
about  various  aspects  of  aircraft  location  and  heading  during  freeze-frames  in  an  ATC  simulation. 
Variants  of  this  technique  were  used  in  subsequent  studies  in  ATC  and  air  combat  simulations  (End¬ 
sley,  1995a;  Fracker,  1990;  Mogford,  1997;  Baumann  et  al.,  1997).  Online  questionnaires  were  used 
in  the  study  described  in  this  report. 


METHOD 


PARTICIPANTS 

The  participants  were  39  people  at  Space  and  Naval  Warfare  Systems  Center,  San  Diego  (SSC  San 
Diego)  employed  as  researchers,  students,  technicians,  or  military  staff.  There  were  nine  active  duty 
military  personnel^  and  36  participants  were  male.  Participants  were  equally  assigned  to  the  three 
conditions  based  on  experience  with  3-D  displays  and  military  symbol  sets.  All  subjects  had  prior 
experience  with  computer  displays.  All  but  five  subjects  had  prior  experience  with  military  symbol 
sets  and  17  subjects  had  no  experience  with  3-D  displays.  Ages  varied  from  20  to  65  with  a  mean 
of41. 

APPARATUS 

Simulations  were  run  on  a  Silicon  Graphics  Indigo  2™  workstation  and  shown  to  participants  on  a 
high-resolution,  wide-format  computer  monitor.  The  simulation  display  dimensions  were  8  inches 
high  by  12  inches  wide.  Probe  questionnaires  were  shown  on  a  second  adjacent  monitor.  Participants 
were  seated  19  inches  in  front  of  these  two  displays,  and  they  viewed  them  with  both  eyes  open  in 
the  dimly  illuminated,  low-glare,  experimental  room.  The  mean  display  luminance  was  approxi¬ 
mately  50  cd/m^.  Participants  interacted  with  the  computer  displays  through  a  computer  mouse. 

Color  printouts  of  the  military  symbols  used  in  the  simulations  were  placed  to  the  left  of  the  appara¬ 
tus. 

PROCEDURE 

There  were  four  phases  to  the  procedure: 

Phase  1.  When  the  participants  arrived,  they  were  screened  for  normal  vision.  Two  tests  were 
given.  They  were  screened  for  visual  acuity  with  a  Snellen  acuity  chart.  We  set  20/40  as  the  criterion 
subjects  needed  to  pass  for  further  participation  (comparable  to  the  California  Department  of  Motor 
VeWcles’  legal  requirement  to  be  able  to  drive).  Participants  were  then  screened  for  normal  color 
vision  with  the  14  standard  Ishihara  color  plates. 

Phase  2.  Participants  passing  the  Phase  1  screening  were  then  trained  on  one  of  three  symbol  sets. 
These  sets  were  the  2-D  non-realistic  symbol  set  (figure  3),  die  2-D  realistic  set  (figure  4),  and  the 
3-D  realistic  symbol  set  (figure  5).  There  are  four  symbols  for  each  platform.  In  the  figures,  plat¬ 
forms  are  grouped  into  four  categories  (ground,  air,  surface,  and  sub-surface).  The  platform  s3mibols 
can  be  one  of  four  colors  on  track  friendliness  (blue  for  friendly,  red  for  hostile,  green  for  neutral, 
and  yellow  for  unknown).  In  addition  to  color,  symbol  shape  varied  by  friendliness  in  the  2-D  non- 
realistic  set  (friendly  blue  symbols  were  circular,  red  hostile  symbols  were  diamonds,  etc.).  Other 
differences  among  the  three  symbols  sets,  apart  from  the  obvious  symbol  shape  differences,  were  that 
there  were  three  extra  air  platforms  for  fighters  (not  actually  used  in  the  scenarios)  in  the  realistic 
sets,  and  the  color  for  “friendly”  was  white  instead  of  blue  in  the  realistic  sets  (to  mimic  the  AADC 
prototype  display  coding). 


^  The  active  military  did  not  score  differently  on  average  from  other  subjects  in  any  of  the  experimental  conditions. 
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You  will  be  asked  to  recall  the  following  symbols. 

Please  take  a  moment  to  learn  what  each  symbol  represents. 
Helptiil  strategies  may  include  concentrating  on  : 
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You  will  bo  asked  to  rocall  the  fbliowing  symbols. 

Please  take  a  moment  to  learn  what  each  symbol  represmts. 
Helpful  strategies  may  include  concentrating  on  : 


Figure  4.  Symbol  set  for  2-D  realistic  condition.  Set  includes  34  different  piatforms  (in  rows)  organized  into  four  track  categories  (grou 
subsurface,  sea  surface,  and  air).  Columns  show  the  four  possible  track  colors.  Color  depicts  friendliness  (unknown,  friendly,  neutral 


Figure  5.  Symbol  set  for  3-D  real  condition.  Set  includes  34  different  platforms  (in  rows)  organized  into  four  track  categories  (ground, 
subsurface,  sea  surface,  and  air).  Columns  show  four  possible  track  colors.  Color  depicts  friendliness  (unknown,  friendly,  neutral,  and  hos' 


We  chose  the  Mil-Std-2525  set  (comparable  to  NATO  symbology)  for  our  conventional  2-D  non- 
realistic  symbol  set  because  individual  platform  types  are  represented  in  Mil-Std-2525.  The  Mil- 
Std-  2525  symbol  set  enabled  us  to  compare  platform  identification  across  the  different  sets.  After 
participants  spent  5  minutes  learning  their  symbol  set,  they  were  tested  on  symbol  identification.  The 
symbol  set  was  replaced  on  the  computer  display  by  a  questionnaire  containing  40  questions,  with 
each  question  asking  about  a  different  track.  The  questions  represented  all  platform  types.  Partici¬ 
pants  were  instructed  to  identify  the  track’s  platform  correctly  from  drop-down  menus.  Time  to  com¬ 
plete  the  questionnaire  was  recorded,  as  were  total  errors.  If  a  participant  failed  to  correctly  identify  a 
track  after  two  wrong  guesses,  that  person  moved  on  to  the  next  question.  When  a  participant 
completed  the  questionnaire,  the  participant  was  given  feedback  on  incorrect  answers  with  a  large 
color  printout  of  the  symbol  set  they  had  just  learned.  This  printout  was  left  visible  to  the  participant 
for  the  remainder  of  the  experiment.  The  participant  was  told  to  use  the  printout  if  a  symbol  was  for¬ 
gotten  later. 

Phase  3.  Participants  were  then  presented  a  practice  simulation.  This  simulation  depicted  a  collec¬ 
tion  of  21  different  tracks  arrayed  on  a  mixed  terrain  map  of  a  peninsula  surrounded  by  water. 
Besides  track  type  and  friendliness,  each  track  possessed  up  to  four  spatial  attributes.  These  spatial 
attributes  were  heading  and  speed  (for  tracks  that  could  move),  and  altitude  and  attitude  (for  air 
tracks).  These  spatial  attributes  were  depicted  in  different  ways  in  the  three  different  display  condi¬ 
tions.  Table  1  shows  the  coding  scheme. 


Table  1 .  Spatial  track  attributes  by  condition. 


Spatial 

Attribute 

2-D  Non-Realistic 

2-D  Realistic 

3-D  Realistic 

Altitude 

Hooking 

Hooking 

Explicit  +  hooking 

Attitude 

Hooking 

Hooking 

Explicit  +  hooking 

Speed 

Leader  length  + 
hooking 

Leader  length  +  hooking 

Leader  length  +  hooking 

Heading 

Leader  orientation  + 
hooking 

Explicit  +  hooking  + 
leader  orientation 

Explicit  +  hooking  + 
leader  orientation 

For  example,  in  figure  2,  tracks  for  a  bomber  are  depicted.  The  bomber  is  of  unknown  friendliness, 
headed  east  at  medium  speed  and  at  low  altitude.  A  yellow  realistic  icon  pointed  to  the  right  depicts 
the  3-D  realistic  bomber  (figure  2,  left)  and  it  is  flying  level.  Directly  under  the  icon  is  a  black  drop- 
shadow  of  a  bomber  on  the  ground  plane  pointed  to  the  right  with  a  medium-length  black  leader  also 
pointed  right.  The  2-D  realistic  bomber  (figure  2,  center)  is  depicted  as  a  realistically  shaped  yellow 
bomber  (seen  from  above)  pointed  to  the  right  with  a  medium  black  leader  off  its  nose.  The 
2-D  non-realistic  bomber  (figure  2,  right)  is  depicted  with  a  Mil-Std-2525B  symbol,  a  "B'  in  a  yellow- 
filled  cloverleaf  with  a  medium-length  black  leader  protruding  from  its  right-hand  side.  To  discover 
the  air  track  altitude  and  attitude  for  conditions  other  than  3-D,  the  user  had  to  hook  the  track. 

Users  hooked  the  track  by  rolling  the  mouse  pointer  over  a  track  to  display  a  text  box  in  red 
that  listed  all  selected  track  attributes.  They  hooked  the  same  way  across  all  three  conditions.  Users 
kept  the  text  box  on  the  display  by  clicking  the  mouse  when  the  text  box  was  present.  This  turned  the 
text  white.  The  mouse  pointer  could  then  be  rolled  away  to  do  the  same  with  other  tracks.  As  multi- 


9 


pie  text  hooks  can  cause  clutter,  participants  could  remove  a  white  text  box  by  clicking  on  it  and  then 
moving  the  cursor  away  from  the  track. 

The  spatial  track  attributes  were  subdivided  into  a  smaller  number  of  ordinal  categories  to  make  it 
both  easier  for  participants  to  remember  them  and  for  us  to  ask  questions  about  them.  Attitude  was 
categorized  as  descending,  level,  or  ascending.  Speed  was  slow,  medium,  or  fast.  Heading  was  N, 

ME,  E,  SE,  S,  SW,  W,  or  NW.  Altitude  was  high,  medium,  or  low.  Appendix  A  discusses  the 
assignment  of  altitude  to  air  tracks. 

During  the  practice  simulation,  a  friendly  missile  was  fired  towards  a  hostile  missile  battery  to  the 
north  of  the  peninsula.  Figure  6  shows  a  screenshot  of  the  beginning  practice  simulation  in  the  2-D 
non-realistic  condition.  The  display  updated  every  4  seconds  and  all  mobile  tracks  moved  by  an 
amount  proportional  to  their  speed  leaders.  Stationary  tracks,  those  lacking  speed  leaders,  did  not 
move. 


O 


Figure  6.  Screenshot  from  beginning  of  practice  simulation  for  2-D  non-realistic  condition. 
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To  assess  SA  during  the  practice  simulation,  every  30  seconds,  the  simulation  froze  and  numbered 
black  circles  randomly  replaced  four  track"^  symbols.  Participants  were  then  asked  about  their  recol¬ 
lection  of  the  four  track  attributes.  On  the  second  computer  monitor,  four  questionnaires  were  dis¬ 
played  corresponding  to  the  four  covered-up  tracks.  Each  participant  had  to  select  what  platform  the 
track  depicted,  its  friendliness  (color),  and  its  spatial  attributes  (altitude  and  attitude  for  air  tracks, 
heading  and  speed  for  moving  tracks).  The  questionnaires  were  presented  as  a  series  of  button  boxes 
that  the  participant  could  fill  in  by  clicking  next  to  an  item.  A  "do  not  know"  box  was  available  to 
click  if  participants  felt  that  they  could  not  recall  any  details,  or  they  could  leave  parts  of  the  ques¬ 
tionnaire  blank.  Participants  were  encouraged  to  fill  in  as  much  of  the  questionnaire  as  possible.  They 
were  urged  not  to  guess  if  they  did  not  know  the  answer.  Appendix  B  shows  a  sample  questionnaire. 

For  the  first  two  probe  sessions  during  the  practice  session,  the  experimenter  gave  the  participant 
feedback  on  the  procedure  of  looking  at  the  simulation  and  then  answering  the  four  questionnaires  to 
facilitate  learning.  After  that,  the  participants  continued  to  acquaint  themselves  with  the  task  uninter¬ 
rupted. 

Phase  4.  After  learning  the  symbol  set  and  participating  in  a  practice  simulation,  subjects  were 
finally  run  in  the  main  experimental  simulation.  This  simulation  depicted  45  tracks  arrayed  on  a 
mixed  terrain  map  of  the  Strait  of  Hormuz  in  the  Persian  Gulf.  In  this  scenario,  a  hostile  Surface- 
to-Air  Missile  (SAM)  is  fired  towards  fiiendly  forces  and  is  shot  down  by  a  friendly  F-14  fighter. 
There  were  19  different  platform  types  represented  in  the  scenario.  Figure  7  shows  a  probe  screen- 
shot  from  near  the  beginning  of  this  scenario  for  the  3-D  eondition.  Figure  8  shows  the  exact  same 
moment  in  the  scenario  for  the  2-D  real  condition  that  figure  7  shows,  and  figure  9  shows  the  same 
moment  in  the  2-D  non-real  condition.  Comparison  between  figures  7  and  8,  therefore,  shows  the 
differences  of  depicting  the  display  in  3-D  or  2-D.  Comparison  between  figures  8  and  9  shows  the 
differences  of  depicting  tracks  realistically  or  non-realistically. 

The  simulation  was  probed  18  times.  It  was  exactly  like  the  practice  simulation  except  that  it  was 
longer  and  the  computer  recorded  response  data.  The  average  time  to  complete  all  four  phases  of  the 
experiment  was  about  1.5  hours. 


*  Tracks  in  the  3-D  condition  had  their  symbols  replaced  by  black  spheres  if  they  were  non-air  tracks  or  they  had 
their  drop-shadows  on  the  ground  plane  replaced  with  black  circles  if  they  were  air  tracks.  We  probed  this  way  in 
order  not  to  ‘tip-ofP  participants  that  we  were  testing  them  on  an  air  track  by  showing  them  a  black  sphere  floating 
in  mid-air. 
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Figure  7.  Screenshot  from  3-D  real  condition.  Camera  angle  was  25°  above  ground  plane.  This  shot 
was  taken  during  SA  probing.  Participants  had  to  recaii  attributes  of  four  tracks  repiaced  with 
numbered  black  circles. 


Figure  8.  Screenshot  from  2-D  real  condition.  Shot  taken  from  exactly  same  moment  in  scenario  as  that 
shown  in  figure  7. 
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Figure  9.  Screenshot  from  2-D  non-real  condition.  Shot  taken  from  same  moment  in  the  scenario  as 
that  shown  in  previous  two  figures. 


DESIGN 

Each  subject  participated  in  only  one  of  the  three  display  conditions:  the  2-D  non-realistic,  the 
2-D  realistic,  or  the  3-D  realistic  condition.  Participants  answered  questions  in  18  probe  sessions. 
Each  probe  contained  four  questionnaires  that  asked  questions  about  six  track  attributes.  Two 
track  attributes  involved  identification  (platform  and  friendliness)  and  four  attributes  were  spatial 
(heading,  altitude,  attitude,  and  speed). 
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RESULTS 


SYMBOL  SETS 

In  Phase  2  of  the  procedure,  participants  were  given  5  minutes  to  study  the  symbol  sets.  We 
timed  their  completion  of  the  questionnaire  that  probed  their  knowledge  of  the  symbol  set  that 
they  just  viewed.  We  also  recorded  the  errors  they  made  on  the  questionnaire.  These  data  are 
summarized  in  table  2. 

Participants  remembered  fewer  of  the  realistic  symbols  than  the  non-realistic  symbols.  A  one¬ 
way  analysis  of  variance  (ANOVA)  revealed  that  the  platform  identification  errors  differed 
significantly  by  display  condition  (F(2,38)  =  6.45,  p.<  .05).  Pair-wise  comparisons  between  the 
mean  platform  identification  errors  revealed  that  participants  who  had  learned  the  2-D  non- 
realistic  symbol  set  made  fewer  platform  identification  errors  than  those  who  learned  the  2-D 
realistic  symbol  set  (g_<  .005).  Participants  were  slower  with  the  reahstic  symbols,  and  more 
friendhness  identification  errors  occurred  in  the  3-D  realistic  condition,  but  neither  of  these 
results  reached  statistical  significance. 


Table  2.  Mean  questionnaire  completion  times,  platform  identification  errors,  and  friendliness  identi¬ 
fication  errors  from  training  sessions  for  three  symbol  sets.  Standard  mean  errors  indicated  (i.e.,  ± 
error). 


Condition 

Mean  Questionnaire 
Completion  Time 
(min) 

Mean  Platform 
Identification  Errors 

Mean  Friendliness 
Identification  Errors 

3-D  realistic 

12.3  ±1.7 

12.5  ±3.0 

1.5  ±0.4 

2-D  realistic 

13.9  ±1.4 

21 .6  ±3.8 

0.6  ±0.2 

2-D  non-realistic 

9.4  ±1.2 

6.4  ±2.5 

0.6  ±0.3 

GROWTH  OF  SITUATION  AWARENESS  WITH  TIME 

Did  3-D  displays  promote  more-rapid  SA  than  2-D  displays?  Figure  10  shows  SA  growth  with 
time,  where  SA  is  defined  as  the  percentage  of  all  correctly  answered  questions  about  all  six  attrib¬ 
utes  of  the  probed  tracks  by  a  given  probe  session.  To  quantify  SA  growth,  polynomial  linear  regres¬ 
sions  (including  linear  and  quadratic  terms)  were  fit  to  the  data  for  the  three  conditions  in  Figure  10. 
Table  3  shows  the  regression  equations.  All  regression  terms  in  table  3  were  statistically 
significant  except  for  the  quadratic  term  calculated  for  the  2-D  non-realistic  condition. 
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Probe  Session 

Figure  10.  SA  growth  for  three  conditions  over  time  (the  18  probe  sessions). 
Percent  correct  for  all  six  track  attributes  over  time  is  plotted  cumulatively  on 
y-axis. 


Table  3.  Regression  equations  and  statistical  significance  of  terms  making  up  regression  equations 
(figure  10). 


Condition 

Regression  Equation 

Significant 

Intercept? 

Significant 
Linear  Term? 

Significant 

Quadratic 

Term? 

3-D  realistic 

5A  =8.4 -1-2.8;? -0.082p^ 

Yes** 

Yes  ** 

Yes** 

2-D  realistic 

5^=142  +  1.5^-0.02/ 

Yes** 

Yes** 

Yes** 

2-D  non-realistic 

5A=21.8  +  0.46p 

Yes** 

Yes** 

No 

**  E  <  .01 


SA  growth  has  two  components.  There  is  the  level  SA  reaches  at  a  certain  time  and  there  is  the  rate 
of  increased  SA  growth  at  that  time.  We  can  conveniently  analyze  these  aspects  of  SA  growth  sepa¬ 
rately  by  using  the  regression  equations. 
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We  get  the  rate  of  SA  increase  with  time  by  differentiating  the  regressions  with  respect  to 
probe  session,  p.  The  2-D  non-realistic  condition  yields 


This  means  that  for  every  probe  session,  p,  SA  increased  by  0.46  percent.  The  3-D  realistic 
condition  yields 

=  2.8-0.164/?^. 

Thus,  SA  increased  by  nearly  2.8  percent  for  every  probe  session  (nearly  six  times  faster  than  for 
the  2-D  non-realistic  condition)  over  the  initial  probe  sessions  (when  the  rate  of  increase  slowed 
as  p  increased).  The  2-D  realistic  condition  yields 

The  rate  of  increase  of  2-D  realistic  SA,  at  initially  1.5  percent  per  probe  session,  was  in- 
between  that  found  for  the  2-D  non-realistic  and  3-D  realistic  conditions. 

We  can  find  out  what  SA  probe  sessions  increased  fastest  for  the  3-D  realistic  condition  over 
the  other  two  conditions  by  solving  simple  algebraic  inequalities.  We  find  that  3-D  realistic  SA 
increased  faster  over  the  first  10  probe  sessions  than  2-D  realistic  SA  by  solving  the  expres¬ 
sion,  2.8  -  0. 1 64p  >  1 .5  -  0.04 p ,  for  p.  Similarly,  3-D  realistic  SA  increased  faster  than  the  2-D 
non-realistic  during  the  first  15  probe  sessions.  Finally,  2rD  realistic  SA  increased  faster  than 
2-D  non-realistic  SA  across  aU  probe  sessions. 

In  summary,  SA  increased  faster  for  3-D,  but  it  did  not  rise  to  higher  levels  than  2-D.  SA 
started  at  a  lower  level  for  3-D  than  2-D.  By  the  last  probe  session,  all  the  curves  in  figure  10 
pinched  together:  3-D  realistic  SA  was  33.8  percent,  2-D  realistic  was  33.6  percent,  2-D  non- 
realistic  was  30.6  percent.  Thus,  SA  starts  at  a  lower  level  for  the  3-D  realistic  display,  grows 
faster  with  time  than  the  other  display  types,  and  saturates  at  a  level  equivalent  to  the  2-D  display 
variants  by  the  end  of  the  scenario. 

TRACK  ATTRIBUTES 

The  previous  analysis  grouped  together  all  the  information  fi'om  the  probe  questionnaires  into 
a  gross  SA  measure.  We  now  examine  performance  for  the  six  different  track  attributes  to  see 
how  they  differed  by  display  type.  Figure  1 1  shows  percent  correct  performance  for  the  six 
attributes  collapsed  across  aU  probe  sessions.  The  attributes  in  figure  1 1  have  been  grouped  into 
those  attributes  that  reflected  information  about  the  identity  of  a  track  (platform  type  and  fiiend- 
liness)  and  those  that  dealt  with  the  spatial  attributes  of  a  track  (altitude,  attitude,  heading,  and 
speed). 


^  Differentiating  the  quadratic  term  of  the  regression  doubles  its  value  when  calculating  rate  of  increase. 
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Figure  11 .  Percent  correct  performance  for  track  attributes  by  condition.  Data  were  collapsed  across 
All  the  probe  sessions.  Error  bars  depict  standard  mean  errors. 


We  ran  a  three-way  split-plot  ANOVA  (six  probe^  x  six  attributes  x  three  display  types)  to  test  for 
performance  differences  across  display  type.  Table  4  shows  the  analysis  results.  Participants  did  not 
differ  significantly  in  their  awareness  of  the  identity  attributes  for  the  different  display  types,  irre¬ 
spective  of  whether  information  was  presented  in  2-D  or  3-D,  realistically  or  symbolically.  The 
spatial  attributes  tell  a  different  story.  Three  of  the  four  were  significantly  different.  Table  5  shows 
how  the  spatial  attributes  were  made  known  in  the  display.  Superior  coding  schemes  are  marked  with 
red  ticks  (I).  Please  note  that  a  user  can  activate  all  spatial  attributes  for  all  conditions  by  hooking  a 
track. 


*  Although  there  were  18  probe  sessions  actually  tested,  because  of  the  quasi-random  process  of  choosing  which 
four  tracks  were  probed  on  a  given  session,  not  aU  attributes  were  tested  on  each  probe  session  (e.g.,  a  probed  oil 
rig  lacks  heading,  speed,  altitude,  and  attitude).  Hiese  missing  data  necessitated  collapsing  adjacent  probe  sessions 
until  all  attributes  were  represented  on  each  probe  level.  Collapsing  three  adjacent  probe  levels  reduced  missing 
data  as  a  proportion  of  total  data  from  19  to  less  than  2  percent  (the  latter  was  replaced  by  appropriate  means). 
Thus,  there  are  six  probe  levels  in  the  reported  ANOVAs. 


Table  4.  Three-way  ANOVA  results:  display  type  by  track  attributes  across  probe  sessions. 


Attribute  Type 

Attribute 

Effect  size  (ti^) 

df 

F 

Significant? 

Spatial 

Altitude 

0.244 

(2,36) 

5.80“ 

Yes 

Spatial 

Attitude 

0.386 

(2.36) 

11.31“ 

Yes 

Spatial 

Heading 

0.180 

(2,36) 

3.95* 

Yes 

Spatial 

Speed 

0.003 

(2,36) 

.05 

No 

Identity 

Friendliness 

0.045 

(2,36) 

.85 

No 

Identity 

Platform 

0.007 

(2,36) 

.12 

No 

*E<.05 

**  fi  <  .01 


Altitude.  There  was  a  3-D  benefit  for  the  altitude  depiction.  This  was  a  substantial  effect  with  per¬ 
formance  four  times  better  for  3-D.  Post-hoc  comparisons  revealed  that  the  3-D  realistic  display  was 
superior  to  both  the  2-D  non-realistic  and  2-D  realistic  (p  <  .05  for  both). 

Attitude.  There  was  a  3-D  benefit  for  depicting  attitude  of  air  tracks.  This  was  a  massive  effect;  per¬ 
formance  was  20  times  better  in  3-D  realistic  than  2-D  non-realistic.  Post-hoc  comparisons  revealed 
that  3-D  realistic  was  superior  to  both  the  2-D  non-realistic  and  2-D  realistic  (p  <  .05  for  both). 

Speed.  Participants’  awareness  of  the  probed  track  speed  in  the  different  conditions  was  not  signifi¬ 
cantly  different.  Speed  was  coded  the  same  way  in  the  different  conditions  by  leader  length  (table  5). 

Heading.  There  was  a  two-fold  benefit  for  depicting  heading  realistically.  Post-hoc  comparisons 
revealed  that  both  the  3-D  realistic  and  2-D  realistic  were  superior  to  the  2-D  non-realistic  (p  <  .05 
for  both).  As  table  5  shows,  heading  was  depicted  by  leader  orientation  in  all  three  conditions,  but  in 
the  realistic  conditions  it  was  also  coded  explicitly  in  the  icon  orientation  on  the  display  (things  sim¬ 
ply  pointed  in  the  direction  they  were  heading). 


Table  5.  Spatial  track  attributes  that  supported  better  performance. 


Spatial 

Attribute 

2-D  Non-Realistic 

2-D  Realistic 

3-D  Realistic 

Altitude 

Hooking 

Hooking 

Explicitl  +  hooking 

Attitude 

Hooking 

Hooking 

Explicitl  +  hooking 

Speed 

Leader  length  + 
hooking 

Leader  length  +  hooking 

Leader  length  +  hooking 

Heading 

Leader  orientation  + 
hooking 

Explicitl  +  hooking  +  leader 
orientation 

Explicitl  +  hooking  + 
leader  orientation 

Red  ticks  indicate  statistically  superior  performance. 
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PERFORMANCE  BY  ATTRIBUTE  OVER  PROBE  SESSION 

We  can  combine  analysis  elements  of  the  two  previous  sections  by  now  asking  how  perform¬ 
ance  varied  for  the  different  attributes  over  a  probe  session.  There  were  six  attributes  times  three 
display  types  across  18  probe  sessions.  Rather  than  show  a  cluttered  graph  with  18  curves,  figure 
12  shows  the  data  with  probe  sessions  collapsed  into  two  groups  (identity  and  spatial  attributes) 
by  condition.  The  figure  shows  that  performance  for  the  identity  attributes  differed  considerably 
from  the  spatial  attribute  performance  over  time.  Participants  became  increasingly  aware  of 
which  tracks  were  depicted.  The  linear  contrast  in  the  three-way  ANOVA  confirmed  this  trend 
(F  (1,36)  =  1 17.40,  p_<  .01).  The  linear  contrast  for  the  spatial  attributes  was  also  statistically 
sigmficant  (F  (1,36)  =  8.30,  p  <  .05),  but  by  a  much  smaller  amount.  Performance  increased,  on 
average,  by  66  percent  from  the  first  to  the  last  probes  for  the  spatial  attributes.  In  contrast, 
performance  increased  127  percent  for  the  identity  attributes.  For  spatial  attributes,  performance 
in  the  3-D  condition  was  significantly  better  (F  (2,36)  =  4.62,  p  <  .05)  than  the  2-D  condition,  in 
part  because  of  better  performance  for  attitude  and  altitude,  as  previously  noted. 


Figure  12.  SA  growth  for  identity  attributes  versus  spatial  attributes.  Performance  shown 
by  probe  session  coilapsed  into  six  groups  of  three  sessions. 
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DISCUSSION 


Our  objective  was  to  test  whether  a  3-D  perspective  display  for  an  AADC  populated  with  realistic 
symbols  would  promote  faster  awareness  of  the  surface  and  air  track  array  attributes  than  an  equiva¬ 
lent  2-D  display.  Our  experimental  design  enabled  us  to  find  out  whether  any  advantages  were 
because  of  display  format  (2-D  versus  3-D),  symbol  realism,  or  both.  In  general,  3-D  perspective 
displays  or  realistic  icons  did  not  work  any  better  than  conventional  2-D  displays  or  symbols.  We 
found  that  3-D  SA  grew  faster  but  it  started  from  a  much  lower  level  than  2-D  SA.  In  fact,  3-D  SA 
took  4  minutes  to  reach  the  start  level  of  2-D  SA.  However,  3-D  was  better  than  2-D  for  some  attrib¬ 
utes.  Likewise,  realistic  symbols  supported  better  performance  for  some  attributes  than  non-realistic 
symbols. 

Altitude  and  attitude  were  remembered  better  in  the  3-D  condition.  Attitude  was  easier  for  partici¬ 
pants  to  recall  when  it  was  coded  by  an  icon  of  a  plane  with  its  nose  up  or  down  and  was  more  diffi¬ 
cult  to  recall  when  participants  had  to  hook  the  track  to  find  out  the  same  information.  This  replicates 
and  extends  the  findings  of  Baumann  et  al.  (1997)  who  had  participants  count  the  number  of 
descending  aircraft  in  a  3-D  tactical  display.  They  found  that  participants  performed  better  using  a 
3-D  display  populated  with  realistic  icons  than  in  a  conventional  2-D  display. 

In  addition,  we  found  that  altitude  was  easier  to  understand  when  coded  in  3-D.  Altitude  was 
coded  explicitly  in  3-D  as  the  distance  between  a  track  symbol  and  its  drop  shadow  on  the  ground- 
plane.  Altitude  information  was  only  accessible  through  hooking  in  the  2-D  conditions.  3-D  perform¬ 
ance  was  surprisingly  good  when  one  considers  that  despite  the  explicit  nature  of  altitude  coding  in 
3-D,  participants  had  two  further  cognitive  steps  involved  in  correctly  gauging  altitude.  First,  they 
had  to  correctly  associate  a  symbol  with  its  drop  shadow  and  then  estimate  the  distance  between 
them.  Second,  they  had  to  compare  this  estimate  with  the  military  altitude  criteria  of  “high,” 
“medium,”  or  “low”  to  get  the  correct  answer  (Appendix  A). 

Realistic  coding  of  heading  was  superior  to  symbolic  coding  of  heading;  heading  is  easier  to 
remember  when  it  is  coded  as  an  icon  pointing  in  the  track’s  direction.  In  the  2-D  non-realistic 
condition,  heading  was  depicted  by  a  leader  pointing  in  the  heading  direction  protruding  from  an 
upright  military  symbol.  Performance  for  the  attribute  of  speed  was  the  same  for  all  conditions.  This 
finding  is  not  surprising  because  speed  was  coded  using  leader  length  for  every  condition. 

An  important  finding  of  this  study  is  that  the  key  to  making  spatial  attributes  known  is  that  they  be 
visually  explicit.  This  explicit  visual  coding  may  come  from  adding  a  third  dimension  (in  the  case  of 
altitude  and  attitude)  or  from  adding  realism  (in  the  case  of  heading).  This  finding  is  consistent  with 
the  expert  who  suggested  that  track  attributes  should  be  displayed  “in  some  other  visual  way  that 
more  intuitively  paints  a  picture”  (Eddy  et  al.  1999). 

There  was  some  indication  from  the  data  (although  not  statistically  significant)  that  symbolic  track 
coding  (i.e.,  identity  attributes)  was  superior  to  realistic  coding.  The  realistic  symbol  sets  were  diffi¬ 
cult  to  learn.  Participants  made  more  errors  and  spent  more  time  answering  questionnaires  about  2-D 
realistic  symbols  than  2-D  non-realistic  symbols.  As  participants  had  been  equated  for  their  subjec¬ 
tive  experience  with  military  symbols,  this  was  not  the  cause  of  the  difference.  Instead,  it  was  more 
likely  that  the  greater  potential  for  confusion  with  the  realistic  icon  set  was  responsible.  For  example, 
when  depicted  realistically,  all  aircraft  icons  have  two  wings  and  a  fuselage,  and  all  ships  have  elon¬ 
gated  structures.  Fine  structural  differences  are  hard  to  detect  among  these  platforms  and  may  not  be 
visible  from  all  viewpoints.  Conversely,  a  non-realistic  symbolic  set  such  as  the  Mil-Std-2525B  can 
be  engineered  for  maximal  discriminability.  The  identification  of  friendliness  was  easier  for  the  Mil 
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Std.  2525  symbols  because  friendliness  was  double-coded  by  shape  and  color  while  in  the  realistic 
sets  it  was  coded  by  color  alone. 

The  participants  also  had  great  difficulty  remembering  tracks  and  track  attributes  from  the 
scenario.  Performance  after  9  minutes  of  viewing  the  scenario  was  about  30  percent  correct  for  all 
conditions.  There  is  a  growing  body  of  perceptual  literature  highlighting  how  little  information  one 
assimilates  from  a  scene  despite  one’s  impression  to  the  contrary.  For  example,  we  are  very  insensi¬ 
tive  to  changes  introduced  into  a  scene  after  blanking  it  out  momentarily  (Rensink,  O’Regan,  and 
Clark,  1997).  MacLeod  and  Willen  (1995)  put  it  best  when  they  noted  that  the  mother  of  all  illusions 
is  the  illusion  of  perceptual  objectivity.  It  appears  that  our  participants  spent  most  of  their  cognitive 
effort  remembering  track  type  and  location,  although  they  were  given  no  instruction  to  do  so  by  the 
experimenters.  Over  time,  performance  for  the  identity  attributes  increased  twice  as  fast  as  the 
performance  for  the  spatial  attributes.  Others  have  found  that  participants  are  most  accurate  at 
remembering  the  identity  and  location  of  air  tracks  shown  in  ATC  scenarios,  with  the  spatial  attrib¬ 
utes  of  heading  and  altitude  being  recalled  worst  (Mogford,  1997).  Perhaps  participants  encode  their 
memories  of  such  scenes  by  identity  first  (e.g.  “there  was  a  fighter  over  there,  now  where  was  it 
going?”). 

In  conclusion,  our  3-D  perspective  display  populated  with  realistic  symbols  did  not  provide  faster 
awareness  of  overall  track  attributes  than  the  equivalent  2-D  display.  Location  and  attributes  of  tracks 
were  learned  quicker  during  the  first  4  minutes  with  the  2-D  top-down  display  with  non-realistic 
symbols  than  with  the  displays  using  the  realistic  symbols  (2-D  or  3-D).  However,  after  9  minutes 
the  2-D  and  3-D  displays  with  the  realistic  symbols  provided  superior  SA  for  track  heading  and  ap¬ 
proached  the  overall  SA  found  with  the  2-D  display  with  non-realistic  symbols.  The  3-D  display  also 
provided  superior  SA  for  track  altitude  and  attitude. 

SUMMARY 

•  Participants  became  increasingly  aware  of  the  identities  of  tracks  over  time,  but  they  had  diffi¬ 
culty  remembering  the  spatial  attributes  of  those  tracks. 

•  2-D  SA  was  initially  2.5  times  higher  than  3-D  SA  and  remained  significantly  higher  for  the 
first  4  minutes. 

•  3-D  SA  increased  six  times  faster  than  2-D  SA  for  the  first  5  minutes. 

•  Overall  SA  was  equivalent  for  2-D  and  3-D  after  9  minutes. 

•  3-D  was  superior  to  2-D  for  the  attributes  of  altitude  (4  times)  and  attitude  (20  times). 

•  Realistic  symbols  were  superior  (two  times)  to  non-realistic  sjmbols  for  the  heading  attribute. 

•  Participants  found  it  harder  to  learn  realistic  symbology. 

We  conclude: 

•  2-D  top-down  display  with  conventional  (non-realistic)  symbols  provided  superior  SA  during 
the  first  4  minutes. 

•  3-D  displays  with  realistic  symbols  provided  superior  SA  for  altitude,  attitude,  and  head¬ 
ing  attributes. 
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RECOMMENDATIONS 

•  Consider  using  2-D  top-down  displays  with  conventional  (non-realistic)  symbols  when  very 
rapid  SA  is  necessary. 

•  Consider  using  2-D  or  3-D  realistic  symbology  when  rapid  heading  appreciation  is  necessary. 

•  Consider  using  3-D  displays  when  rapid  altitude  and  attitude  appreciation  outweighs  other 
factors. 


In  the  introduction,  we  reviewed  the  relative  merits  of  2-D  and  3-D  displays.  Based  on  that  review 
and  the  experience  we  gained  during  this  project,  we  offer  the  following  broad  recommendations: 

•  Perspective  view  displays  should  be  used  judiciously  for  tasks  that  show  significant  user  S  A 
performance  enhancements  because  absolute  certainty  about  two  dimensions  is  lost  by  adding 
in  the  third  dimension. 

•  Consider  using  both  a  2-D  and  a  3-D  display  of  the  same  information  to  optimize  the  benefits 
of  each. 

•  Consider  improving  the  2-D  display  with  explicit  visual  coding  of  attributes  without  hooking 
to  get  pop-up  text  boxes,  which  is  time  consuming.  An  important  consideration  in  the  choice 
between  2-D  and  3-D  is  accurately  locating  a  platform  in  time  and  space,  which  is  a  highly 
critical  attribute  that  was  not  evaluated  for  the  3-D  display  in  this  study.  2-D  top-down 
displays  with  conventional  (non-realistic)  symbols  provide  accurate  perceptions  of  platform 
location. 

The  3-D  display  engineering  could  improve  in  two  other  ways:  (1)  make  the  realistic  symbols 
more  discemable,  and  (2)  provide  recommendations  for  users  on  how  they  might  maneuver  their 
viewpoint  in  3-D  to  minimize  poor  perception  of  far  distances  and  the  clutter  effects. 

Our  future  research  will  focus  on  perception  accuracy  of  3-D  platform  location  using  drop  lines, 
shadows,  and  relative  size  as  cues.  The  outcome  of  this  follow-on  study  could  strongly  affect  any 
recommendation  for  introducing  3-D  displays  into  the  AADC  console. 
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APPENDIX  A 


The  air  tracks  were  classified  as  high,  medium,  or  low  as  an  altitude  function  (feet)  in  a  realistic 
military  context.  For  example,  a  bomber  at  a  14,000-ft  altitude  is  considered  low  although  it  is 
physically  higher  than  an  attack  helicopter,  which,  at  6000  ft  is  considered  high.  This  military  context 
was  explained  to  participants  with  a  poster  depicting  the  graphic  (for  3-D  realistic)  condition  in 
figure  A-1. 


Figure  A-1 .  Altitude  was  coded  in  a  track-dependent  way  for  military  realism.  Graphic  shown  to 
participants  in  3-D  real  condition. 


APPENDIX  B 


Figure  B-1  shows  a  screenshot  of  the  questionnaire  that  participants  filled  out  on  the  computer 
to  indicate  their  awareness  of  probed  track  attributes.  In  this  case,  the  participant  is  filling  out  the 
responses  to  the  first  probed  item  (one  out  of  four).  The  participant  has  described  the  track  as  an 
air  track,  specifically  a  bomber,  of  unknown  friendliness  (Force  ID),  at  a  high  altitude,  with 
medium  speed  and  level  attitude  heading  south.  When  the  participant  has  completed  as  much  as 
possible  for  each  of  four  questionnaires,  the  “submit  all  screens”  button  (bottom  right)  is  pressed 
and  the  scenario  is  resumed  until  the  next  probe  session. 


Figure  B-1.  Screenshot  of  questionnaire  to  indicate  awareness  of  probed  track  attributes. 
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